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ABSTRACT 

A correlation between recall (the retrieval of all 
available relevant documents) and quantity of text which served as a 
source of index terms on input can reasonably be expected. 
Specifically, recall should decrease as the quantity of text serving 
as a source of index terms is restricted. On the other hand, the time 
for indexing and therefore the input cost shoulc be less, thus 
establishing a tradeoff between input cost and retrieval 
effectiveness. It was desired to quantify the effect of restricting 
the source text on both retrieval effectiveness and input cost. An 
experiment was designed in which the full technical documem text was 
divided into five categories: (1) title; (2) abstract; (3) table of 
contents and lists of figures and tables; (4) author-assigned 
keywords; and (5) the body. An experimental data base of technical 
documents was created, for which the index term source category and 
the time required for indexing by category was recorded. Sets of 
Selective Dissemination of Information (SDI) and retrospective 
searches were run against the data base, and retrievals were 
analyzed. Based on the results, it was decided that the body of the 
document could be excluded as a source of index terms. This decision 
was translated into a reduction of unit cost from $10 to $8.25. 
(Author/SJ) 



ERIC 



FILMED FROr-l BEST AVAILABLE COP'/ 



us /fcr RT^ EN . of hEai I H ' 
EOL'tA -ION i WEltARE 

OFf ir- OF tDllCATiQN , ' 

This, docl /iENT m4, (.i.'< u, .. 

DUCED EXACTLY Ab >• •' 

. .. __ __ THE PERSON ORi-,. ■ " 

ArfV\L-TR-73-53 inating it poiNib oi » .-. 

IONS STATED DO Nvi' '.■ 
^—4 REPRESENT OFFICIAL (•■ ■ 

CATION POSITION OR . 

o 

^ THE USE OF SELECTED PORTIONS OF TECHNICAL 

DOCUMENTS AS SOURCES OF INDEX TERMS AND EFFECT 
ON INPUT COSTS AND RETRIEVAL EFFECTIVENESS 



H. H.SCHUMACHER 
J. F. MARCH 
F.L.SCHEFFLER 

UNIVERSITY OF DA YTON RESEARCH INSTITUTE 
DA YTON, OHIO 



TECHNICAL REPORT AFML-TR-73-53 



CD 
rH 

o 
o 



Approved for public release; distribution is unlimited 



AIR FORCE M/TERIALS LABORATORY 
AIR FORCE SYSTEMS COMMAND 
WRIGHT PATTERSON AIR FORCE BASE, OHIO 



NOTICE 



When Governtnent drawings, specifications, or other data are used for 
any purpose other than in connection with a definitely related Government 
procurement operation, the United States Government thereby incuro no 
responsibility nor any obligation whatsoever; and the fact that the govern- 
.nent may have formulated furnished, or in any v%^ay supplied the said 
drawings specifications, or other data, is not to be regarded by implication 
or otherwise as in any manner licensing the hclder or any other oerson or 
corporation, or conveying any rights or permiGsion to manufacture, use, 
or sell any patented invention that may in any v/ay be related thereto. 



Copies of this report should not be returned unless return is required by 
security cons ider .itions , contractual obligations, or notice on a specific 
document. 

AIR FORCE/56780/31 May 1973 — 400 



THE USE OF SELECTED PORTIONS OF TECHNICAL 
DOCUMENTS AS SOURCES OF INDEX TERMS AND EFFECT 
ON INPUT COSTS AND RETRIEVAL EFFECTIVENESS 



H. H. SCHUMACHER 
J.F.MARCH 
F.L.SCHEFFLER 



Approved for public release; distribution is unlimited 



FOREWORD 



This report was prepared by the University of Da^-ton Research 
Institute, Daytijn, Ohio, under Air Force Contract F33615-71-C- 1069, The 
work described herein was accomplished under Project 7381 "Materials 
Application" and Task 7 38103 "Materials Information Development, Collection, 
and Processing. " The effort was administered under the direction of the 
Operations Office of the Air Force Materials Laboratory with Mr. Edward 
Dugger (AFML/DO) as Project Monitor. 

This is a final summary report and covers the work accomplished 
from I December 1971 through 30 November 1972. 

The authors acknowledge the efforts and contributions of Mr. Eugene 
R. Egan, Mr. Mark S. Klug, Mr. John T. Logan, Mr. Donald L. Wannemacher 
and Miss Virginia C. Weber in providing indexing input and assistance in 
compiling statistical data. 

This technical report has been reviewed and is approved. 




Operations Office 



Air Force Materials Laboratory 



ii 



ABSTRACT 



A ctjrrelation between rerall (the retrieval of all available relevant 
documents) and quantity of text which served as a source of index terms on 
input can reascjnaljly be expected. Specifically, recall should decrease as 
the quantity of text serving as a source of index terms is restricted. On the 
other hand, the time for indexing and therefore the input cost should be ^ess, 
thus establishing a tradeoff between input cost and retrieval effectiveness. It 
was desired to cjuantify the effect of restricting the source text on both retrieval 
effectiveness and input cost. An experiment was designed in which the full 
technical document text was divided into five categories: I, title; 2, abstract; 
3, table of contents and lists of figures and tables; 4, author-assigned keywords; 
and 5, the body. An experimental data base of technical documents was created, 
for which the index term source category and the tirrie required for indexing 
by category was recorded. Sets of Selective Disemination of Information (SDI) 
and retrospective searches were run against the data base, and retrievals were 
analyzed by category in terms of retrieval response, S; relevant document re- 
sponse^ R; categorical relevance, R; indexing time, T; and retrieval efficiency, 
E and E, It was found for the subset of documents retrieved for all searches, 
that 81% of the available relevant documents were retrieved from categories 
1-4, whereas the indexing time required for these four categories was only 53% 
of the total indexing time, as compared to the time for all five categories. For 
the entire set of documents input into the experimental data base, the portion 
of indexing time for t'le first four categories was 60%. Based on these results, 
it was decided that the body of the document could be excluded as a source of 
index terms. This decision was translated into a reduction of unit cost from 
$10 to $8. 25. 
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SECTION I 



INTRODUCTION 



The work described in this report covers \ vo aspects: the operations 
and nature of the Aerospace Materials Information Center (/^MIC) and an 
experimental program to test the effect of the source(s) of inaex terms upon 
the retrieval effectiveness and input cost. A brief description of AMIC is 
given followed by considerations affecting the input to and retrieval from the 
system. The specific purpose of the experimental program was to provide 
quantitative data upon .vhich a management decision could be made regarding 
the tradeoff between input cost and retrieval effectiveness. 

1. I DESCRIPTION OF THE AMIC SYSTEM 

The Information Systems Section of the University of Dayton Research 
Institute (UDRI) has established and presently maintains and operates a 
document retrieval system in support of the Aerospace Materials Information 
Center (AMIC). The document retrieval system operated by the University 
of Dayton contains approximately o5, 000 documents concerning materials 
research and development with new acquisitions being processed continually. 
The establishment, modification, and operation of the document retrieval 
system are described in references 1 through 9. The present report describes 
the work performed from 1 December 1971 through 30 November 1972. 

The AMIC document retrieval system has been in operation with retro- 
spective search capabilities since 1963. The purpose of the system is to 
provide scientific and technical information to qualified recuesters in a 
timely and efficient manner. The information is supplied in the form of 
abstracts of documents pertinent to the search request; these abstract formats 
also contain complete bibliographic information, including AMIC access 
number, DDC AD number or NASA N number, generating agency, report 
number, title, author, contract number (if applicable), and date of issue of 
the docunient. The documents themselves are available from the Materials 
Documentation Center (MIC) maintained at the Air Force Materials Laboratory 
(AF'^L). Hardcopy documents are available on loan to AFML requesters. 
Microfiche documents are reproduced and the duplicate microfiche are pro- 
vided to the requester if permanent retention is desired. 

The AMIC document retrieval system is primarily concerned with the 
materials aspect of technical documents. Because of the concentration on 
materials, retrieval capabilities irom t materials standpoint are very com- 
prehensive. Retrieval can be quilc speiific. For example, a request for all 
information on the alloy Aluminum 2024 T6 can be readily satisfied; on the 
other hand, retrieval can be general in nature, e.g., high temperature fatigue 
of all metals and alloys. Similarly, ^quester could ask for information on 
boron reinforced Epon epoxy composites or for aircraft structural applications 
of any composite material. 
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Retro'-^pocti v'e sea^rches enci^nipassing the entire range of materials 
informatK)!) aro run regularly by UDRI in response to requests from the AFM L. 
To cnsrre that the rcfiuestor receives abstracts which are relevant to the 
re juest, aJi abstracts and index cards retrieved are screened for content by 
a I DRI infc>rmation specialist to assess their relative pertinence to the originally- 
started request - 

AMIC al.^o offers SD^ services. SDI refers to Selective Dissemination 
of InformatifHi, which is the practice of providing timely, pertinent references 
to documents in particular areas of interest to a number of users, each user 
receiving only material of potential interest to him. The concept is also 
referred to ?s current awareness. The SDI program is based on the periodic 
input of document index data to the AMIC system. 

1. 2 INDFXI\G PHILOSOPHY 

There are a number of viewpoints regarding the indexing of technical 
documents. For purposes of this discussion we will restrict our consideration 
primarily to coordinate indexing consisting of manual assignment of Keywords 
or descriptors to serve as the set of retrieval access points for that document. 
We will not involv^e indexing by classification schemes. The topic we wish to 
address is the "depth** of indexing. 

It is necessary at this point to define and differentiate ''depth'* of indexing 
from "specificity" of indexing, since these terms are often used interchangeably. 
By depth of indexing we mean the extent of the docximent which serves as a 
source of index terms; this can vary from only the title to the entire full text 
of the document. By specificity we mean the degree to which an index item is 
described in relation to the hierarchical possibilities for its description. To 
illustrate, let us consider the title "Fatigue Properties of Aluminum 2024-T6. " 
Indexing from the title would be specific (including detailed specific nomencl:;ture) 
but "shallow" (only the title was used as the indexing source). To be sure, 
there is a correlation between the depth of indexing and the specificity of 
indexing, but the two concents are different. 

The philosophical question regarding indexing is: How deep is deep 
enough'^ Tl|g Keyword in Context (KWIC) index which was developed by 
H. P. Luhn depends on the display of significant words appearing in the title. 
Often only the title and abstract are used for manual indexing. A number of 
automatic indexing systems depend on the extraction of significant words from 
only the title and abstract; from this pr?::tice one could infer that these sources 
are considered adequate for indexing. Many people contend, however, that the 
full text of the document must serve as the source of indexing. 

We believe that the answer to the question must be based on the scope 
of the information system and on the nature of the information reeds of the 
u.^er. All too often, in our view, the indexing function has governed the depth 
of indexing with too little attention being given to the actual user needs. Surely, 
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the user needs should have some influence on how deeply and .o\v specifically 
material is indexed in the first place. In the case of AMIC, which is specialized 
in the area of materials, there is a real need for highly specific retrieval 
regarding materials as evidenced by the types \if requests presented by the 
user. Therefore this specificity in retr v ibility should be provided o • 

input. In recognition of the need for spc .ty and r ?alizin^^^ '^e corr*^' ition 
of depi-h and specificity of indexing, UDRI has used the full tc.^t of the document 
as a source of index terms. 

However, with some years of im^exing experience, LDRI has de ?lopea 
intuitively the notion that indexing time probably could be substanti ily reduced 
without serious loss of retrievability by restricting the source of index terms 
in documents to certain well-defined portions thereof, but no. \i eluding t.ie 
body. 7'his idea had never been subjected to rigorous testing un'iJ the 
experimental program reported herein. 

Miller^^ in studying the MEDLARS system concluded that a large r- L-rt- 
age of index terms can be found in the document title. He cited the lOUowing 
evidence: 

(a) 228 le^al documents showed that 64% of the titles contained 
all the i\.dex terms 

(b) 5 titles from the Physical Review contained 63% of the total 
number of index terms 

(c) 4 titles from Chemical Abstracts Subject Index contained 57% 
of the total number of index terms 

Depth of indexing in terms of titles and abstracts has been studied by 
Tell , in Sweden. He reported that titles and abstracts are good sources 
of index material based on relevance judgments of the users. Tell further 
suggests (perhaps wistfully) that the authors' Knowledge that indexing will be 
accomplished from the title and abstract will cause them to write more 
informative titles and abstracts, 

13 

Lancaster states that with regard to cost effectiveness, an indexing 
system can be improved either by: (a) altering the indexing/ retrieval language 
in such a way that system costs are reduced while the present level of search 
effectiveness is maintained; or (b) making system changes that improve search 
effectiveness with no measurable increase in overall system cost. As stated 
earlier, a primary concern of UDRI was to improve the cost effectiveness of 
AMIC. 
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1. 3 UDRI APPROACH TO THE PROBLEM 



a 
was 



The approach taken by UDRI in its experimental study represents 
variation of alternative (b) as proposed by Lancaster. An experiment wc 
designed to test the effect of the depth of indexing on retrieval effectiveness 
of the AAIIC system. According to Tell studies have shown that exhaustiv. 
indexing will diminish retrieval effectiveness. There is a point when an 
increase of indexing depth will decrease retrieval quality. He goes on to 
state that indexers are more consistent when induing from titles and/or 
abstracts than from full text. One of our studies also suggested this 
phenomenon. 

The hypothesis assumed by UDRI for the AMIC experiment is as 
follows: 

Lidexing time, and therefore cost, will be substantially reduced 
^vithout serious loss in the number of relevant documents retrieved 
if the source of index terms is limited to exclude the body of the 
document. 

A model was designed to test this hypothesis. The data base for the model 
consisted of 984 documents indexed by three experienced indexers. Each 
indexer indexed about 330 documents. The indexers were instructed to select 
and designate terms derived from each of the five different sources within the 
documents. In designating the indexing source, only the additional terms 
obtained from each subsequent section were indicated; thus the categories 
were considered cumulative and not completely independent. These srurces 
and combinations thereof represented varying depths of indexing. The five 
sources of index terms were categorized as follows: 

Category 1 Title 

Category 2 Abstract 

Category 3 Table of Contents, and lists of Tables and Figures 

Category 4 Author Keywords 

Category 5 Body of document -5^ 

Index terms were coded as to the source (Category 1, Category 2, etc.) 
by using these code numbers on the index card. The code number for each 
term thus provided the key to determine the source of the term. The indexers 
also maintained records of the indexing time required for each category. Two 
sets of actual previously run searches (97 SDI searches, 15 retrospective 
searches) were run against the model data base. Each search was then 

* Note that the body of the document is the narrative descriptive portion of 
the document exclusive of the titlr, abstract, table of contents and author 
keywords. The full text is represented by 1-5. 
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analyzed according to the number of documents retrieved as a result of the 
five indexing categories. From the data obtained, retrieval effectiveness 
was determined as a function of the depth of indexing and input cost was 
correlated with the depth of indexing. 
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SECTION II 



EXPERIA^ENTAL PROGRAiM 



£.1 RETRIEVAL RESPONSE AS A FUNCTION OF THE SOURCE OF INDEX 
TERMS 

The fir: t part of this experiment addressed itself to the problem of 
determining the amount of raw retrieval or recall as a function of indexing 
depth. No effort was made in the analysis of these results to determine 
relevance. The documents retrieved on a given search contained not only 
the usual access number but also the category number, thus revealing the 
indexing source within the document by which the retrieval was made. By 
referring to Appendix A, it can be seen that some documents on a given search 
were retrieved in all five categories. For example, on SDI Search #99019, 
Document #69882 was retrieved by each of the five categories. In this 
particular example, any one of the five sources of index terms or any 
combination thereof would have been sufficient for retrieval. Other document 
were retrieved by only a single category. For example, on SDI Search #99071, 
document #200355 was retrieved only by Category 5. In later analysis, this 
document was judged to be relevant. In this example, evidently it was 
necessary to index from the body of the document to effect retrieval. The 
two exam.ples cited above represent the extreme cases; most documents were 
retrieved with various combinations of Categories I through .5. A tally was 
made (see Appendix A) for each search showing the total num.ber of documents 
retrieved and the percentage of the total number of documents first retrieved 
by each category. For example, if a document was retrieved on Category 2, 
then indexing from the title and abstract was sufficient to effect retrieval. 

For the purpose of this experiment, all searches were assumed to retrieve 
all available documents, that is, retrieval obtained from full text indexing 
represented 100% recall. We here introduce the term "retrieval response. " 
Retrieval response represents the ratio of the documents retrieved by a given 
category to the entire set of documents retrieved from full text. The retrieval 
response we will symbolize byS. Figure I shows the retrieval response for 
the various categories. The retrieval responses are presented for each 
category such that any given document is counted only for that category by 
which retrieval first occurred. The corresponding averaged data are presented 
in Table I. Raw data are provided in Table A-1, Appendix A. 
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TABLE I 



* RETRIEVAL RESPONSE AS A FUNCTION OF 

THE SOURCE OF INDEX TERMS (CATEGORY) 



Category 


Retrieval Responses S(%) 


Cumulative Retrieval 
Response VS(%) 


1 - Title 


19 


19 


2 - Abstract 


43 


62 


3 - Table of Contents 
and List of Figures 
and Tables 


9 


71 


4 - Author keywords 


2 


73 


5 - Body 


26 


100 
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Figure 1. Retrieval Response as a Function of the Source of Index Terms. 
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An analysis of the retriex al response data reveals that the title alone 
docs not yield a substantial portion of AMIC documents for the set of searehe. 
run against the data base. Addition of the abstract as a source of index terms 
provides an additional 43% of the documents retrieved froi-. full text indexing; 
thus abstract and title together provide 62^^ of retrieval response. Interestingly, 
author ke^-Avords represent only a 2^o improvement over title, abstract, and 
table of contents and list of figures and tables. If the body ' ' » docun ^t ^s 
excluded, a retrieval response of 73% is obtained. 

2. 2 RELEVANT DOCUMENTS RESPONSE AND CATEGORICAL RELEV V : x 

AS A FUNCTION OF THE SOURCE OF INDEX TERMS 

The second part of the experiment was concerned with the relevance of 
the retrievals. In particular, we were interested in the distr'bution of relevant 
documents among the five categories. From the retrieval response we learne<^^ 
that if the body is excluded as a source of index terms, 27% of the document: 
which would have been retrieved from full text indexing were, in fact, not 
retrieved. But of this 27%, how many were actually relevant Obviously, 
if none of the 27% were relevant, we would be actually improving the retrieval 
effectiveness by not retrieving nonrelevant documents, while at the same time 
reducing input cost since the time for indexing the body cf the document would 
no longer be required. Such an idealized situation would represent an increase 
in relevance with no corresponding reduction in recall. Actually we expected 
some loss in recall as the price for increased relevance. 

In order to provide some answers, an analysis was performed to determine 
the relevance of the documents being retrieved. The searches run against the 
model data base were screened by judging the document to be either relevant 
or nonrelevant. Two experienced UDRI AMIC information specialists performed 
this task. The searches were screened independently, each information 
specialist screening adifferent set of searches. A few of the searches were 
screened by both specialists; no significant differences were observed. After 
screening, the relevant documents for each search were summed in order to 
determine both the distribution of relevant documents by category and the 
relevance factor for each category. 

From the relevance data, we can consider relevance in various ways. 
One possibility is to consider the relevance as the ratio of number of relevant 
documents in a given category to the total number of documents in that category. 
This concept is defined as the ^'categorical relevance, symbolized by R. Another 
way is to consider the ratio of the number of relevant documents retrieved in 
a given category to the total number of relevant documents available. We define 
this concept as the "relevant document response, " symbolized by R. The 
relevant document response indicates the distribution of relevant documents by 
category. 2) R represents the usual concept of relevance, i.e., the number 
of relevant documents retrieved divided by the total number of documents 
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retrieved, JJhc relevant document response (iw by category and the categorical 
relevance (R) a\v» shown in Figures 2 and 3. The corresponding averaged data 
are given m Tabic IL Raw data are provided in Table A-Z in Appendix A, 
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Figure 2. Relevant Document Response as a Function of the Source of Indexing Term 
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TABLE II 



UF:r.EVANT DOCUMENT RESPONSE AND CA'. FGOiUCAL RELr.'VANCK 
A FUNCTION OF THE SOURCE OF IM:)EX TERMS (CATEGORY) 



Category 


Relevant Document 
Response R 


Categorical 
RoIe\ ance R 


CuTiiulc*^live 
Value:. 5]R 


I 


25% 






2 


47% 






3 


7% 


5 8" ■ 




4 


2% 






5 


19% 


3b% 




Xl-5 


100% 







The standard definition of relevance is tlie rf>1i<j of th.. iiu iat^t-: ri^lovant 
documents retrieved to the total nuirjber of retrieve'] d n- u . v "t^. r ':>r 2]^"^' 
i.e. Z R, the relevance is 50% 

Analyzing the data, it can be seen tliat R i 5 -Toaloot lor ihp title (65'}.)), 
but this value varies considerably for the other eaic pories. The oiprmlicance 
of these data is that the index term sources are not particularly g v-^d dis- 
criminators as far as differentiating nonrel^vant from relevant runients, 
although the general trend is toward lower R as the category increases. It 
should be pointed out that retrieval by the author keywords represents a high 
categorical relevance as would be expected. If one re-ordored the categorie:? 
such that the categories correspond to the quantity t^f tekt tlierein, a relevance 
pattern would emerge such that as the quantity of text used as a source of index 
terms increased, the relevance would decrease. This is shown in Figure 3. 

The implication of R is that one is about as likely to itnss relevant docxmients 
by not indexing from the body of the document ars by n it indexing from the table 
of contents, and list of figures and tables- Indexing froai the title, author key- 
words and abstracts, on the other hand should result in a good probability of 
retrieving relevant documents. The idealized sitxiation described earlier in this 
section is not approximated in practice- 



13 



The relevant docunioni rec:f> uoo on i.)v )'\ : -i. ^ • » - • n 

sliowing the distribuLioii ol r'^ievanl docaiuc i> k * , i i ^ l . « - 

One can see that 25% of the avai labie robj ik ci '^'v.; rL'/ir-, ; i,,, vJ 

on indexing from the title alone, wIjC vt-^as fully ■ w-* i .'.^ a k jIv i-, m « . 
documents were retrieved fron. title c'nd ab'Ur? l i.\vu>iii^. L.el'i-. ^ i v ■ 

f X om the table of contents and list '>t (inures snr ^^*hi . i ^< a'-i.^d rot 
of relevant documents from 7Z^/o to 7^''fo; additional i ^uo- mil. iroii\ .p.- LmT,/ t.;.- 
document resulted m furtlier retrieval of VT\ tb^ r.^OilAbte ide^ am doctMi-.nr- 
wliich Vv'ould have been lost if the- body had not beer, 'isc u a source oj ird.jy.in^; 
terms. If we assume that all available relevant uoLunients wjre i.-lru-^ c-d 'iic i 
the full text served as the source of index teriiJiJ, \{ ,i Intc'-e. n K) plo. l: - 
cumulative relevance and recall resulting froni iii<. ve^-. n^ d^. uih ^ icu. i m.;, 
because this shows that increasing the depth ot' indexing increases re^ajl to 
a lesser degree than might be intuitively aiiti cipat c d, as ib .-.h.jun m \ i^v.ic 

2. 3 INDEXING TIME AS A FUNCTION OF TJiE SOl^RCK uF IaIDK : i Ei^XiS 

The time required to index a document is directly rcialed to the co^t of 
indexing. It is important to consider the retrieval effectiveness as a luucticn 
of index term source in terms of the time, and therefore cost, required for 
indexing from the various sources of terms, Fron^ the precedinp, di^^ :us?ions, 
we h-ave seen that R and S are distributed as foilow^: 





S{%) 




Category 1 


19 


25 


Category 2 


43 


47 


Category 3 


9 


7 


Category 4 


2 


2 


Category 5 


26 


19 



The body of the document (Category 5) represents the largest portion of text 
of the document. Therefore, at the outset, one could easily hypothesize that the 
major portion of the indexing time would be spent on the body. Yet the above 
results clearly indicate that index terms froin the body of the do.ruinent were 
responsible for only 26% nf all the documents retrieved and only 197? of the 
relevant documents retrieved. The question yet to be answered is: what is 
the distribution of indexing time by category ' 

The answer to this question can be derived from an analysis of the data 
on indexing time for each category. These data were acquired by having each 
indexer record the stopwatch time required to index each docuiiient by the five 
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categories. By knowing wiiich du-. uHiL-iits and c rro^ip • id i-aKu' i'l^ • ^''ctc- 

retrieved on the sear dies, it was a rei ai.cly t .1-3 y nutttcr to dt u r.-mK tiu- 

distribution of indexing tin.e h) dtc<^or\ pritMUL']^ tnat ^ao - I uiiUMit:^ 
actually retrieved by the sear.:h(j^. 

To facilitate handling thu Hiri«.x time daia and .rrc^alinu th». search output 
with index time, keypunch cards were preparer! f .r cacl: d w.uj. C xi i i ilie deita 
base. A print-out from the keypunched c^dta was ul^tamod ii^aia^ d. c an^ejits in 
order by access number. Froni Ihe scare ii prm^f-ut shc^t:^, .jtul indexing 
time per retrieved document wc^s ibic incd tor each scKir,h '! abU- /V-3). 

This process was repeated [or et.ch Uwcnmrnt roiri?\«.G fur aii arc *ic o , The 
result of tlie summation of induxint; luiie by atc-'i^u ry (ov <*uc ss. <ir^n is .-)ht)wa 
in Table A- 4. Table A- 4 siiows the dist ributifMi of tt>lal indexirij, tiuie t.)r the 
documents retrieved on a particular ht-arclj anioi^c; tlie ^ anou.- c ai; ,i?,!)rie^. Table 
III indicates the distribution of time spent in indexmi^ tln^-,*' f!oi cntb retrieved 
for the set of SDI searches rune The indexing time syiiiljujJized 1 . These 
data are shown graphically in Figure 4. 



TABLE III 

DISTRIBUTION OF TIME SPENI IN INDEXING TilOSE DOCrMKMS RETRIEVED 
FOR THE SDI SEARCHES RUN ON THE EXPERIMENTAL. \)M\\ BJ^SE 



Catego ry 


Time Spent in Indexing 
T {%) 


Cumularive Values 


1 


3 


3 


2 


36 




3 


1 1 


50 


4 


3 


5 3 


5 


47 


100 



We now have data on retrieval response (S), relevant docunu-nt response 
(R), and indexing time (T) by category. We ran therefore determine the dis- 
tribution of these values and make some inft^rences about tne desirability of 
selective indexing, i.e., indexing by selected categories. P'^igure 5 shows the 
trend by cumulative category for all of these values. 
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LEGEND 

□ INDEXING TIME (T) 

CUMULATIVE INDEXING TIME (IT) 



36 



T] 



i 

\ 

/ 
/ 
/ 
/ 
/ 



39 



II 



50171 

I 

/ 



/ 



53 



/ 
/ 
/ 

\ 

/ 



100(71 



DEPTH OF INDEXING (CATEGORY) 

Figure 4. Distribution of Indexing Time by Category for the Documents 
Retrieved for the SDI Searches Run 
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It is interesting to consider the retrieval efficiency for various sources 
()f i-idcx term So The retrieval efficiency, E, is defined as the percentage of 
total doc unients r^neved divided by the per^'cntage of the time required for 
i-tdexing, i„ e. ^ relevant document efficiency, E^is defined iS the 

percentage of rele\ ant documents retrieved divided by the percentage of the 
time required for indexing, i, e. , E - — , These \'ahies ire ^ - n in Table 
IV. 

TABLE IV 

RETRIEVAL EFFICIENCY (EJ AND RELEVANT DOCUMENT EFFICIENCY 

(E) BY CATEGORY 







R 


C ategory 


T 


E ■-- T 


1 


6. 34 


8. 33 


Z 


1. 19 


I. 30 


3 


0. 82 


0. 64 


1 

1 ^ 

i 


0. 67 


0. 67 


1 ^ 
1 


0. 55 


0.40 



From the above consider'.tions we can state that indexing from the title is 
by far the most efficient, and that indexing from the body is rather inefficient. 
The reader is warned, however, that these values are interdependen t rather 
than independent, in other words, the values for Categories 2-5 are dependent 
on the results from the preceding category. Our experiment really shows the 
added value achieved when additional portions of the documents are used as 
sources of index terms. The results would be somewhat different if the categories 
liad been considered ab mutually exclusive, completely independent entities 
serving as sources of indexing terms. However, our experimental design was 
formulated to indicate the additive efficacy of the sources of index terms. Our 
indexing pliilosophy and procedures follow a pattern such that one would always 
start with the title as a source of indexing terms and only then proceed to the 
abstract to determine additional index terms and thence to the table of contents 
and list of figures and tables for yet additional terms, etc. 



17 



T 1 ^''(i tHU' pur i.i\u'ir\ study v.'< naaloa , ivestigatc i:crtaiii ph^'tiomena 
wnlr-'i 'uc' -.'-ihly ^'^fhit ik iV^'* rf^^ ults, Tlie data reported up to thi 5 point 
ro ^vur' :'D[ scaril'C's, \ lu- i.^'tMrt* of SDl searoht^s is siuh that they tend to 
/..'-irf' ^-^ 1.'-' r-'^'i ndturo tb } rt-t x >-pcctive nearclics, Theie^iore, since titles 
ii'd .-.'j)sL ra^ 1 s , i -p?'^, -nilv^ te !.d ■ ')C more een<'^ra]ly ^- ^criptive text il ^ the 
Dodv >i ih'^ rc'p jrt lU^'.'Ii', v'. : d that the lesiJts roni r^'C -irche 'ild 

iriid to ta.or the hvs dctadeci uidc-xi that necessarily would occur fror; --ing 



n]o3 r'lnd cibe'trcu' ; .is ndi^x \crm sources* It was our original intent to test 
! 2 • 1 ( u' 1 " t r ' ' • 7 < ^t 1 \ a n I ''^ 0 1 c :i ^ • ' h. f , ^ I • ) v. e v f ^ r . th e d at a ba s e v -^s s s ni i\ ^ ' 
('MO d.jrai^tLTi^: ^ if .'ft vi v \' few --etr-rj^cct ^'o searc'^*-". l>jid r^ny letrit n ils. We 
did oh: Jin data for fifteen ospe cli ve seardics. The-sc uata are presen ^d 
m "^'phle Tht.^ r?\v dei!.! are gwcn ui Tables A -^5 to A- 7 in Appendix A, 



CON^PARISO:! OF' R, S, AND 1 FOR RETROSPECTIVE VS, ^^DI 

SEARCHES 



C at'?lir;r ' 



3 
4 



R 



SI3I 



Kelx' 





2S 






n 






7 






2 






19 


3 

1 . 



R 



SDT 



65 
53 
38 
59 
36 



Retro 



100 
64 
33 
50 
80 



SDI 



19 
43 
9 
2 
26 



Retro 



5 
54 
5 
3 
33 



SDI 



3 
36 
1 1 

3 
47 



Retr: 



I 

42 
8 
2 

48 



Fii-n iho^c data it can bf seen that the times for inde..ing are virtually the 
?arne lor both retrnspiu tivc and SDI searches. The values for S, R, and R show 
signi ficant di ffc;rej'C<;.'. Interestingly, the document response, S, for the body 
of the document i j about the same for both SDI and retrospective searches, but 
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the r^atopuri a' role\ancc, R, inrreases from 36% to 80%. Also, percentage of 
rele\ ant d xmi-ii t.'nts rc+ric'.*od by Category 5 indexing increases from 19% to 
39%. It ^lioukl br rocoQnized tliat the data from retrospective searches is far 
less extendi- * ti^an fr-r the SDJ soarches, due both to the lower number of 
relrast>f^c^' ■ ( iu-^'cAicb vi.n a^iu fewer retrievals per retrospective search. 
Neverlhc'lcss, tlu trcncK^ and differences are definitely significant. Table VI 
shews the rr"u^ n y fMrt ^rs Ic r retrospective and SDI searches. 



TABLE VI 

CcrvlP A^lSOX C;F F and E for retrospective and SDI SEARCHES 







E 


_ s 

' T 






E 


_ R 
T 


Category 


















SDJ 




Retro 




SDI 


Retro 


1 




6. 3 4 




5.00 




8. 33 


7.00 


2 




I. 19 




L 28 




1. 30 


1. 19 


3 




0. n?. 




0.63 




0. 64 


0.25 


4 




0, 67 




1. 50 




0. 67 


1.00 


S 




n, 5^ 




0.69 




0. 40 


0. 81 



Another factor wc wished to explore was the indexing time distribution 
for new, relatively inexperienced indexers vs. that for highly experienced 
iiidexers. '^v;prrted that the inexperienced indexer would tend to spend a 

particularly lonj^ time on indexing the body of the report. One new indexer at 
UDRI was selected aiid instructed to record time-category data for 50 documents 
just as the i!«dcxcrs lu the primary experiment had done earlier. The results of 
the time di^tnliutifm iuv experienced and inexperienced indexers is shown in 
Table VIL 
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TABLE VII 

DISTRIBUTION OF INDEXING TIME BY CATEGORY FOR EXPERIENCED 

VS. INEXPERIENCED INDEXERS 



Category 


Experienced* 
{%) 


Inexperienced 
{%) 


1 


13 


11 


2 


38 


20 


3 


10 


11 


4 


6 


6 


5 


32 


53 


Total time/Doc 


24 min 


47 niin 



'J^For all documents indexed; note difference between these valuers and T. 



2.5 INPUT COSTS AS A FUNCTION OF THE SOURCE OF INDEX TERMS 

An important consideration is the unit cost of input to the AMIC system. 
The indexing process represents a major portion of the costs of the operation 
of the AMIC system. Therefore any saving in the unit cost of input processing 
becomes significant for cost reduction when considered in terms of the number 
of documents added annually. A previous technical report indicated that the 
cost for processing a report into the AMIC system is about $10 including clerical 
processing aind indexing. The cost of indexing has been held down by using 
University students as paraprofessional indexers. These students are thoroughly 
trained in indexing by a student indexer training program developed'^ and validated 
by the University of Dayton. 

The cost of indexing (not including clerical processing) amounts to about 
$6. 50 per document. From Table VII we can see that from 32% to 5 3% of stop- 
watch indexing time is expended in indexing \ising the body of the document as a 
source of indexing terms. The stopwatch time is not altogether realistic for 
determining actual indexing time, since there is a certain document handling 
time over and above the reported stopwatch time for Indexing froiri the various 
portions of the document. Previous expeiience suggests that about one-third 
of the time required for indexing is document handling tiir.e. If \\e assume that 
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appioximately one third ot tho cost nf indexing is taken up .n the r.ieclianio^ of 
document handhng, then the cost corresponding to the actual stopwatch hu'exuig 
time about $4, 33. From Table VII it is known that the indexing tiiiie pci 
document can be reduced by 32% to 53% depending on the ie^el of experience 
of tlie indexer, -A rodsonnble approximation of the average :s 40^a Tf v/e 
assume that the bf^dy of the docunient is eliminated as a source of index terms, 
then the cost of indexing (apart truHi document handling) can be reduced fron. 
^4. 33 to S^.60, rcpre:.enring a saving of $1. 73 per document. Considering an 
annual ii.put oi jOOO (o 8000 docum.ents per year it can be seen thai an annual 
saving of $8, 650 to $13, 840 can result; this saving can be transformed into 
additional input in*:o the system. Simply by eliminating the body as a source 
of index terms the init cost of documents entered into the system ^vould be 
reduced from $10 to S^. ZS. 

2.6 conclusion:; 

An important puioose of this study was to assist us in a decision making 
situation. Specifically, we knew that for our system we could restrict the 
portions of the documents which would serve as sources of index terms, and thus 
reduce indexing time. As shown in the preceding section, this factor can be 
translated into reduced unit inpit cost. In order to make a decision concerning 
desirability of restricting the source of indexing terms, we needed experimental 
evidence to indicate the effect of i-stricting the index term sources. These effects, 
not only on indexing time, but also m retrieval effectiveness, needed to be 
determined. The data presented above provide us with the information we 
needt.^d. 

It should be pointed oui: that the v^ata acciuired in our experimental program 
apply to the A^4IC system, and therefore t>e specific data and the analysis there- 
of may serve as guidelines for other situations and systems, but a precise 
transfer and application of our experience into another environment should not 
be expected. Also the concept of relevance must be approached with caution. 
Our previous experience^ and the experiences of others have shown tendencies 
toward inconstancy and inconsistency of relevance judgments. 

We found that the amount of time required for indexing from the first four 
categories amounted only to about 5 3%, whereas the tiiDC required for the body 
amounts to 47%. Yet for SDI searches, the required extra indexing effort results 
in an increase of relevant documents of 19%; for retrospective searches, an 
increase of 39% results, based on a limited number of retrospective searches. 
Thus there is a trade-off between the cost of input and retrieval effectiveness, 
i.e. , the ability to retrieve the relevant documents. The tradeoff is more 
dramatic with SDI searches than with retrospective searches. It is interesting 
to note that the title and abstract alone achieves a rele^^ant docun^ent response 
of 72% with an indexing effort of only 39%. By increasing the indexing effort 
also to include the table of contents and list of figures and tables, the relevant 
document response is increased to 81% with an additional indexing expenditure 
of 14%. 
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The figures lifted abo\ c ruler to Sl)» -t-.i r 
spective searches, the tradeoff betv\tcn lefrie al U < : 

effort IS not nearly as ;i.trong. However, one nivu.* >i , . 
of retrospective searches and SDI scartht-^. SI.'l rc b 
with each update^ Rctruspe. tive bear, h nctiML\ viMi \ jv 
v^hile the SDI program has increased in the uwnA t :* (j<r:. 
there is not necessarily a cause - and-eifect rel.ii ^ 

In view of all the above factors, ve hcavt: • Ui-j. ^ 
reasonable to use the first four categories as source i 
to include the body of the report. By doin^ \k'c vsk -> - 
able relevant documents, but we save 47% of the >;idcx il, . 
based on retrieved documents. Based on Llie inciexjtii, m 
(not just the subset retrieved), the indexing tuiu saMuv^ , 
experienced indexers and 53% for inexp'cTi encod iiKU^xer^. 
are more appropriate, since in practice all docuinentr, are 
we could not predict the subset of documents which would b.. 
searches. 

Figure 6 shows a composite of results of)nsidt 
of iOO retrieved documents, and con^paring retrieval '^ffecti 
5]1 - 4 with the additional retrieval provided by Catc^r .ry ^ , 
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CATEGORIES 
21-4 



CATEGORY 
5 




TOTAL OF 100 

DOCUMENTS 

RETRIEVED 



CATEGORIES 
SI-4 



y///A 

DOCS ^ , 


32 \ 
DOCS ^ 


y////. 





INDEXING TIME OF 53% 
PRODUCES 74 DOCUMENTS, 
OF WHICH 42 ARE RELEVANT 



CATEGORY 
5 



17 DOCS 




ADDITIONAL INDEXING TIME 
OF 477o PRODUCES AN 
ADDITIONAL 26 DOCUMENTS 
OF WHICH ONLY 9 ARE 
RELEVANT 



Figure 6. Retrieval and Distribution of a 100-Document Set, Comparing 
Categories 2)l-4 with the Additional Retrieval Effected by 
Category 5. Shaded areas represent relevant documents. 
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SECTION III 



AiMIC SYSTEM OPERATIONS 



3.1 SUMxMARY OF THE AMIC SYSTEM 

In the normal course of document processing, AFiM L technical reports; 
documents on automatic distribution from AEC, DDC, NASA, and FTD; and other 
Government R&D publications of the Army, Navy, and Government contractors 
such as Westinghouse, The Boeing Co. , the University of California, etc. a^^* 
received at the UDRrs off- campus Materials Documentation Center, BIdg. i , 
AFML. After checking for duplication and screening for relevance of the contents 
to the mission objectives of AFML, identifying accession numbers are assigned. 
The documents are then delivered to the University of Dayton campus. 

At the University of Dayton the documents are indexed and abstracted by 
information scientists whose technical disciplines enable them to select pertinent 
keywords from a thesaurus displaying acceptable keywords. The indexed data 
fr om the documents is transferred to keypunch cards and then convcrced to mag- 
netic tape for storage on the CDC 6600 computer in BIdg. 676 at WPAFB. The 
documents themselves are returned to the Materials Documentation C nter, BIdg. 
17, WPAFB for storage. 

Retrospective searching for information requested by AFML personnel 
proceeds as follows: the request is made directly to an AMIC information 
specialist, either in person or over the phone. The Project Leader is present 
at WPAFB in Building 17 tw 3 days per week to take such requests. At other 
times, the contact can be made by phone. The search request is assigned to 
the appropriate information specialist for the formulation of an appropriate 
search strategy. This search data is provided to the CDC 6600 computer 
facility for a batch mode corriputer search of the data bank. Access numbers 
whose index data qualify them for retrieval according to the search strategy 
are retrieved. Access numbers, which identify documents, are printed out 
and returned to UDRI, where a file of abstracts is maintained for screening 
by the information specialist. Relevant abstracts are copied and sent on to the 
Materials Documentation Center for distribution to the AFML Requester. 

The SDI searches are run periodically against the update data. In pre- 
paring an SDI profile, the AFML user discusses his subject request of continuing 
interest with an AMIC information specialist, through personal interview*;. The 
statement of interest is processed into an SDI profile and the search data are 
prepared on magnetic tape for running on the CDC 6600 computer. The computer 
does a search of current update data only (information addeii-f o the data bank 
in the current and two previous years). Document access numbers correspond- 
ing to the SDI profile are used to select abstracts for copying. These abstracts 



25 



^ '^^ ' ' ' ' '* ' " '^'l' ^t."tN i^DcunvM-it alien Center for distribution to the 
SDI use. , • . . ! i.ani' is jpdat.''i approximately every six weeks. AH of the 
P^^^^' ' ^ . ^ ' .'rf prr^^onted in the fnrrn of charts and flow diagrams 

3. '^1 • t v.^i'i, ;,\' the amic 

• ' 1 i* ■ ' i']Mi)ve the efficiency and effectiveness of the overall UDRI 

\\\\.<\ r.il r. '>rj^a:-i»7'jt.on of the AMIC project vyas implemented during 

'^^ ''''^ ' r ' I- < -jrdlnnt^' t^c activities of the off-campus Materials 

'^^ ' • M:3inta:neri the AFML more closely witli those aspects 

-'f ' — ' ro pcrfc>rKK-d o:i-campus. Specifically, the off-campus 

clerivPl o» r If MM brought under the supervision of the UDRI Information 

Syr^p^i S,. ^:i .;o^ ^ lericrd Supervisor, thus bringing all clerical operations of 

ihc A'sljr -v-w ' '.'cdier ^ir^V-r 'jnr^ head. 

'^'"'^^ • ' '•'•ta^f ' [inve acc rued as a result of the reorganization. 

ri< rl: pj' » . ^ ,1 . ..id c: )cumentati(m metliods liave been made uniform for both 
(>n- r-nd . > i]; . > o[n rat]ons. Clerical personnel from each operation became 
mi;ro 1 iS'. l.i: V tiK* ^,ct'-Tties jf the other. The effect of the reorganization 

i\w \' r..-,|.^ i \ Ttcri'i^ Infornvation Center is to provide a more coordinated 
unit . 

In an effort to enii. nc^ and expand the SDI program, a number of personal 
intervin-.vs w^ ro hrld v/ith already active SDI participants as well as potential 
ncv/ client- i ^ oifcr\iev.s with the already active participants, feedback was 
oblaii.cd reun^ .f ])< ^'no appropriateness of the abstracts distributed corresponding 
to the rrh files. With the experience of having received abstracts 

ovrr a p» vi «d i 1 i -r.K , it was 'fteu possible to pinpoint specific retrieval terms 
in t. p nroL m: v\hif ti . ere cau-sing nonrelevant retrievals. In many other caoe^ 
il was po-:o;]ilr t r,-: ;: rfain terms which could be negated in order to suppress 
nonroJevanl r< cr^c vals^ A number of profiles were modified to incorporate 
the se chanrro<^, Td «.')me ca-cs profile modifications were made to reflect 
change?? in Ine -uhjc:i area? o^ interest of some individuals. These redirections 
in subject interest usually cc -rie about due to the phasing out of certain projects 
and the initiation of new projects, or because of personnel reassignments. 

A number of referrals to other AFML persons who might be interested 
were made by r^ctjv SDI users. These referrals were followed up with inter- 
views and new STH users were obtained by this method. By the end of the 
reporting pcrioc? ilicre wrre 166 active SDI participants. A list of SDI users 
is p ese.ited m \ppendix B, and the SDI topics are given in Appendix C. 
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3.4 INPUT 



During the period covered by this report, 1 Dec 71 through 30 Nov 72, 
5704 documents were indexed and processed into the system. Of this number, 
59 were handbooks, 139 were state-of-the-art reports, 58 were bibliographies, 
and 44 were symposium proceedings or papers. The documents were indexed 
with an average of 20.6 terms per document (exclusive of automatic generic 
postings) with an average indexing time of 32. 3 minutes. There are now 65, 467 
documents in the AMIC document retrieval system. The distribution by subject 
category is given in Table E-l. The subject category definitions are shown in 
Appendix D. 

3. 5 SEARCHING 

A total of 99 retrospective technical requests \^ere processed by the 
Information Systems Section during the report period. An average of 24. 3 
abstracts was printed per search for forwarding to the search requesters. A 
list of retrospective search topics is given in Table E-2, 



ERLC 
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TABLE A-l 

J^ETRIiLVAL RESPONSE (S) BY IXrrxiNG SOL^'^F CArEGORY 



Search No, 


Total Docuiricnts 


Retrieval Response S j 




Retrieved 


Category 






1 


2 




4 


5 


99005 


23 


3 


13 


; 


1 




99006 


23 


3 


7 


; 3 


0 


1 

' 10 


99019 


102 


37 


43 




1 


16 


99022 


18 


2 


12 


0 


0 


4 


99024 


31 


13 


9 


i 4 


' 1 


4 


99045 


19 


> 6 


7 


1 

! 3 


1 


2 


99046 


10 


4 


6 


: 0 


1 0 


0 


99047 


4 


I 


2 


0 


0 


1 


99070 


4 


0 


4 


0 


0 


0 


99071 


19 


4 


8 


4 


0 


3 


99075 


62 


18 


35 


3 


2 


4 


99077 


34 


1 


13 




1 


17 


99080 


18 


6 


7 


4 


0 


1 


99082 


5 


0 


3 


1 


0 


1 


99083 


8 


0 


5 


1 


0 


2 


99085 


5 


2 


2 


0 


0 


1 


99087 


2 


1 


1 


0 


0 


0 


99088 


1 


0 


1 


0 


0 


0 


99197 


19 


7 


9 


2 


0 


1 


99199 


7 


3 


4 


0 


0 


0 


99200 


1 


0 


0 


0 


0 


1 


99201 


9 


1 


2 


2 


0 


4 


99203 


75 


6 


21 


9 


0 


39 


99204 


4 


0 


2 


1 


0 


1 


99215 


5 


2 


1 


0 


2 


0 


99218 


6 


1 


2 


3 


0 


0 


99219 


26 


7 


10 


0 


0 


9 


99220 


8 


1 


4 


1 


0 


2 


99221 


20 


0 


4 


5 


1 


10 
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TABI>E A-I continued 



Search No. 


Total Documents 
Retrieved 




Retrieval Response 


s 






Category 




1 — - 






1 


2 


3 


4 


5 


99222 


7 


1 


3 


0 


1 


2 


99223 


7 


0 


6 


0 


0 


1 


99224 


14 


0 


8 


1 


0 


5 


99225 


74 


11 


40 


6 


1 


16 


99226 


5 


1 


1 


2 


0 


1 


99227 


19 


1 


7 


5 


0 


6 


99228 


17 


0 


12 


1 


0 


4 


99229 


9 


0 


0 


0 


0 


9 


99231 


3 


0 


3 


0 


0 


0 


99232 


32 


10 


11 


4 


0 


7 


99234 


4 


0 


3 


1 


0 


0 


99235 


6 


0 


3 


1 


0 


2 


99236 


11 


2 


5 


1 


0 


3 


99237 


34 


6 


10 


7 


0 


11 


99238 


35 


27 


5 


2 


1 


0 


99239 


19 


5 


9 


3 


1 


1 


99241 


42 


1 


17 


6 


1 


17 


99278 


63 


12 


27 


5 


3 


16 


99286 


50 


0 


19 


5 


0 


26 


99288 


23 


8 


12 


1 


0 


2 


99289 


44 


19 


14 


2 


1 


8 


99290 


5 


0 


4 


0 


0 


1 


99291 


21 


4 


11 


5 


0 


1 


99292 


13 


3 


4 


2 


2 


2 


99353 


3 


0 


0 


0 


0 


3 


99354 


3 


0 


1 


1 


0 


1 


99355 


1 


1 


0 


0 


0 


0 


99357 


1 


0 


0 


0 


0 


1 


99361 


3 


3 


0 


0 


0 


0 
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TABLE A-1 continued 



Search No, 


Total Documents 




Retriev 


al Response S 






Retrieved 


Category 






1 


2 


3 


4 


5 


99365 


74 




26 


9 


1 


37 


99 366 


21 


0 


9 


2 


0 


10 


99367 


17 


3 


8 


1 


0 


5 


9936B 


51 


7 


24 


9 


0 


LI 


99369 


14 


1 


4 


4 


0 


5 


99374 


19 


4 


12 


0 


0 


3 


99376 


51 


7 


23 


3 


2 


16 


99377 


12 


1 


4 


2 


0 


5 


99 378 


258 


54 


131 


18 


3 


52 


99379 


1 


0 


0 


0 


1 


0 


99380 


8 


0 


4 


1 


1 


2 


99385 


12 


2 


6 


1 


0 


3 


99386 


3 


0 


2 


1 


0 


0 


99387 


3 


0 


2 


0 


0 


1 


99389 


1 


0 


1 


0 


0 


0 


99 390 


1 


0 


1 

X 


0 


0 


0 


99391 


2 


1 


I 


0 


0 


0 


99 392 


5 


0 


2 


0 


0 


3 


99394 


1 


0 


0 


0 


0 


1 


99395 


1 


0 


1 


0 


0 


0 


99397 


28 


5 


12 


2 


2 


7 


99399 


2 


0 


1 


0 


0 


1 


99401 


7 


2 


1 


0 


0 


4 


99402 


90 


10 


40 


4 


1 


35 


99403 


2 


0 


2 


0 


0 


0 


99405 


4 


0 


0 


2 


1 


1 


904. " 


3 


0 


0 


0 


0 


3 




26 


5 


11 


2 


0 


8 


99413 


1 


0 


1 


0 


0 


0 
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TABLE A- 2 

RELEVANT DOCUMENT RESPONSE (R) AND CATEGORICAL RELEVANCE 
(R) BY INDEXING SOURCE CATEGORY 



oearcn iNo, 


i otai uocuments 


Retrieval Response R 




Retrieved 


Category 






i 


2 


3 


4 


5 




1 7 
It, 


c 


8 


1 


0 


1 




1 A 


3 


3 


3 


0 


7 


Qon 1Q 


?A 


0 


10 


1 


1 


6 




0 


1 


3 


0 


0 


2 


QQ(\2a 


17 
1 1 


9 


6 


0 


0 


2 




Q 

7 


5 


4 


0 


0 


0 




0 


3 


3 


0 


0 


0 




1 

i 


1 


0 


0 


0 


0 




u 


0 


0 


0 


0 


0 


Qom 1 
77U * 1 


i f 


3 


7 


3 


0 


4 




0^ 


18 


35 


3 


2 


4 


QQf\T7 


^ f 


1 


9 


3 


1 


13 


c\c\fy on 
yyuou 




5 


3 


4 


0 


0 


no n 0 9 




0 


2 


1 


0 


0 


yyuoj 


A 


0 


3 


1 


0 


0 


yyUoD 


9 
C 


1 


1 


0 


0 


0 


99087 


1 


1 


0 


0 


0 


0 


99088 


1 


0 


1 


0 


0 


0 


99197 


9 


3 


4 


2 


0 


0 


99199 


3 


2 


1 


0 


0 


0 


99200 


0 


0 


0 


0 


0 


0 


99201 


3 


1 


1 


0 


0 


1 


99203 


16 


4 


6 


1 


0 


5 


99204 


2 


0 


2 


0 


0 


0 


99215 


4 


2 


1 


0 


i 


0 


99218 


4 


1 


2 


1 


0 


0 


992^" 


10 


6 


2 


0 


0 


2 


99220 


3 


0 


2 


1 


0 


0 


99221 


10 


0 


5 


1 


1 


3 
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TABLE A-2 continued 



Search No, 


Total Documents 
Retrieved 


Retrieval Response R 


Category 






1 


2 


3 


4 


5 


99365 


42 


1 


23 


4 


0 


14 


99366 


10 


0 


5 


0 


0 


5 


99367 


9 


1 


5 


1 


0 


2 


99368 


10 


7 


3 


0 


0 


0 


99 369 


10 


1 


3 


3 


0 


3 


99374 


13 


3 


8 


0 


0 


2 


99376 


25 


7 


10 


2 


1 


5 


99377 


3 


0 


1 


1 


0 


1 


99378 


121 


32 


60 


8 


3 


18 


99379 


0 


0 


0 


0 


0 


0 


99380 


7 


0 


4 


1 


1 


1 


99385 


5 


0 


4 


1 


0 


0 


99386 


1 


0 


1 


0 


0 


0 


99387 


3 


0 


2 


0 


0 


1 


99389 


1 


0 


1 


0 


0 


0 


99390 


1 


0 


1 


0 


0 


0 


99391 


2 


1 


1 


0 


0 


0 


99 392 


3 


0 


2 


0 


0 


1 


99394 


I 


0 


0 


0 


0 


1 


99395 


1 


0 


1 


0 


0 


0 


99397 


18 


1 


11 


0 


1 


5 


99399 


0 


0 


0 


0 


0 


0 


99401 


1 


1 


0 


0 


0 


0 


99402 


58 


8 


27 


1 


2 


18 


99403 


0 


0 


0 


0 


0 


0 


99405 


1 


0 


0 


0 


1 


0 


99408 


5 


0 


0 


0 


0 


5 


99410 


15 


4 


7 


1 


0 


3 


99413 


1 


0 


1 


0 


0 


0 


99415 


10 


0 


5 


0 


0 


5 
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TABLE A-2 continued 



Search No, 


Total Documents 


Retrieval Response R 




rvetri e veG 




Catego ry 










1 


2 


3 


4 


5 


99416 


1 


0 


1 


0 


0 


0 


99421 


3 


0 


2 


0 


0 


1 


99422 


3 


0 


2 


0 


0 


1 


99424 


5 


2 


2 


0 


0 


1 


99430 


40 


11 


18 


4 


0 


7 


99432 


1 


0 


1 


0 


0 


0 


99434 


17 


8 


6 


1 


0 


2 


99436 


7 


4 


3 


0 


0 


0 


Rel. Docs, R 


991 


248 


461 


70 


20 


190 


R(%) 




25% 


47% 


7% 


2% 


19% 




50% 


65% 


5 3% 


38% 


59% 


3b% 
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TABLE A- 4 

niSi RIBUTION' OF TIME SPENT INDEXING OF THOSE DOCU.\!ENTS 
KETKIEVED ON THE SEARCHES RUN 



Search 
No. 


Docs 
Rut rieved 


Summation uf Indexing Time 


by Indexing; 


Source Category 


I 


2 


3 


4 




Total 


9V00S 


23 


-i 
J 




12 


10 


37 


104 


99006 


23 


L. 
O 


A 1 


c 

cb 


1 


8b 


16 b 


99019 


102 






A A 


6£ 


396 


912 


99022 


18 


c 




3 


0 


66 


126 


99024 


31 


10 


1 A "J 


^4 


15 


105 


263 


99045 


19 


Q 

7 


DU 


"J c 


12 


36 


142 


99046 


10 


7 


7 


r\ 
U 


1 


19 


64 


99047 


4 


c 


1 A 


O 


2 


6 


27 


99070 




u 




0 


0 


9 


2^ 


99071 


19 


7 
1 


/ U 


o 1 


4 


87 


189 


99075 


62 


7 7 




12 Q 
OO 


39 


297 


747 


99077 


34 


2 




19 
JC. 


8 


194 


344 


99080 


18 


Q 

7 




28 


11 


85 


213 


99082 


5 


0 


15 


6 


0 


18 


39 


99083 


8 


0 


21 


4 


1 


17 


43 


9908S 


5 


3 


17 


0 


1 


15 


30 


99087 


2 


2 


8 


2 


2 


4 


18 


99088 


1 


0 


6 


0 


0 


5 


11 


99197 


19 


11 


78 


29 


11 


86 


215 


00100 


7 


5 


24 


5 


1 


31 


66 


99200 


1 


0 


0 


0 


0 


8 


8 


99201 


9 


2 


18 


19 


3 


51 


93 


99203 


75 


9 


139 


71 


2 


319 


540 


99204 


4 


0 


9 


2 


0 


5 


16 


99215 


5 


4 


22 


0 


3 


11 


40 


99218 


6 


2 


26 


18 


0 


14 


60 


99219 


6 


11 


74 


3 


6 


51 


145 


99220 


8 


4 


13 


7 


1 


13 


38 


99221 


20 


0 


.!9 


1 


1 


43 


74 
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TABLE A-4 continued 



— — 

Search 
No. 


Docs 
Retrieved 


Summation of Indexing Time by Indexing Source Category 


1 


2 


3 


4 


5 


Total 


99222 


7 


1 


18 


0 


1 


1] 


31 


Q077 ^ 


7 


0 


27 


5 


0 


17 


49 


00774 


14 


0 


53 


0 


0 


68 


121 


QQ77^ 


74 


16 


2 30 


66 


2 


251 


565 


007 Ify 

7 7*- /-O 


c; 


2 


9 


22 


3 


28 


64 


QQ777 

7 7*- *- » 


1 7 


2 


37 


34 


2 


83 


158 


7 7 *-*- 0 


17 


0 


55 


5 


0 


58 


118 


0Q770 


0 

7 


0 


0 


0 


0 


81 


81 


QQ7 '^l 

77*- Jl 




0 


18 


0 


0 


16 


34 


Q07 ^^7 


^7 
^*- 


16 


115 


41 


3 


112 


287 




I 


0 


15 


15 


0 


17 


47 


Q07 "^j; 




0 


15 


3 


0 


28 


46 


3d 


1 1 


2 


4^^ 


8 


0 


45 


98 


77^ J # 




6 




7 


3 


101 


131 






31 


137 


48 


24 


91 


331 


QQ "> 70 


1 Q 
I7 


8 


70 


33 


9 


67 


187 


99^4l 


42 


2 


134 


64 


9 


190 


399 


99278 


63 


17 


138 


42 


15 


191 


403 


99286 


50 


0 


88 


55 


3 


244 


390 


99288 


23 


15 


119 


32 


24 


99 


289 


99289 


44 


34 


181 


42 


12 


117 


386 


99290 


5 


0 


26 


0 


0 


13 


39 


99291 


21 


5 


90 


26 


1 


71 


193 


00202 


13 


4 


20 


7 


3 


51 


85 


99353 


3 


0 


0 


0 


0 


13 


13 


993^J 


3 


0 


7 


3 


0 


7 


17 


9935"^ 


I 


0 


0 


0 


0 


0 


10 


99357 


1 


0 


0 


0 


0 


1 


1 


99361 


3 


4 


19 


0 


3 


11 


37 



45 



TABLE A~4 continued 



Search 
No. 


uo cs 
Retri eved 


oun 


nmation of Indexing Tmie by Indexint 


: Soii.ce Cateeorv 


1 


2 


3 


4 


1 

! 5 


Total 




74 


2 


207 


106 


5 


■) — ^ . 

1 

1 42Q 


749 


99366 


21 


0 


59 


17 


0 


iI6 


192 


99367 


17 


5 


59 


26 


17 


101 


218 


QQ 

77 JOO 


C 1 

DL 


7 


178 


57 


22 


266 


5 30 


QQ ^AQ 
77 j07 


1 A 

14 


c 


18 


9 


3 


45 


77 


OQ 7*7 A 
77 J / 4 


1 n 

19 


10 


104 


32 


24 


117 


287 


Q Q '7 "7 Z. 
77 J /O 


C 1 

Dl 


12 


183 


49 


13 


298 


555 


QQ ^^77 


1 "y 


Z 


28 


24 


10 


72 


136 


QQ a-? Q 
77 J 1 O 


9 t Q 


95 


966 


200 


73 


1218 


2552 


QQ '?7Q 
77 J / 7 


1 
1 


0 


0 


0 


1 


5 


6 


QQ OOn 
77 jOU 


Q 
O 


0 


18 


7 


1 


27 


53 


99 3oD 


12 


5 


47 


14 


4 


59 


129 


QQ OQiL 
77 JOO 




0 


9 


2 


0 


9 


20 


Q Q O 0*7 

993o7 


3 


0 


9 


0 


0 


22 


31 


QQ AO 

77 J07 


i 


0 


4 


0 


0 


2 


6 


99390 


1 


0 


5 


0 


0 


4 


9 


99391 


2 


4 


4 


4 


0 


8 


20 


9^392 


5 


0 


11 


5 


0 


10 


26 


99394 


1 


0 


0 


0 


0 


5 


5 


99395 


1 


0 


4 


0 


0 


2 


6 


99 397 


28 


10 


103 


30 


12 


107 


262 


99 399 


2 


0 


4 


10 


0 


17 


31 


99401 


7 


3 


13 


0 


1 


53 


70 


99402 


90 


16 


257 


61 


14 


434 


782 


99403 


I 


0 


5 


0 


0 


2 


7 


99405 


4 


0 


0 


11 


2 


23 


36 


99408 


3 


0 


0 


0 


0 


6 


6 


99410 


25 


6 


69 


12 


4 


103 


194 


99413 


1 

# 


0 


4 


0 


0 


5 


9 
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TABLE A~4 continued 



Search 


Docs 


Summation of Indexing Time by Indexing Source Category 


No. 


Ret neved 


1 


2 


3 


4 


5 


Total 


99415 


13 


0 


51 


9 


0 


63 


123 


99416 


2 


0 


10 


0 


0 


8 


18 


99421 


11 


2 


44 


0 


0 


63 


109 


9942Z 


8 


0 


40 


7 


0 


53 


100 


99424 


6 


4 


30 


0 


3 


29 


66 


99430 


49 


22 


218 


81 


31 


239 


591 


99432 


8 


0 


28 


4 


1 


21 


54 


99434 


33 


27 


137 


34 


11 


104 


313 


99436 


12 


5 


60 


8 


2 


49 


124 


Time 


1996 


590 


6447 


1860 


574 


8400 


17, 871 


Dist nbutio 
by 

Category 

(%) 




3% 


36% 


11% 


3% 


47% 


100% 


Time/Doc = 
10 m in* 
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TABLE A-6 

RELEVANT DOCUMENTS RETRIEVED FROM THE EXPERIMENTAL 
PATA BASE FOR RETROSPECTIVE SEARCHES 



Search No, 


Totsil iJocumcnts 


Retrieval Response 




Retrieved 


C ate^ory 






I 


2 




4 


c 

D 


12603 


9 


1 


5 


0 


0 


3 


12629 


0 


0 


0 


0 


0 


0 


12630 


2 


0 


1 


0 


0 


I 


12636 


0 


0 


0 


0 


0 


0 


22634 


5 


0 


2 


0 


0 


3 


22535 


12 


2 


6 


0 


1 


3 


32528 


2 


0 


1 


0 


0 


I 


32604 


1 


0 


0 


0 


0 


I 


32605 


3 


0 


2 


0 


0 


I 


42543 


5 


0 


3 


I 


0 


I 


42637 


2 


0 


0 


0 


0 


2 


62504 


0 


0 


0 


0 


0 


0 


92594 


1 


0 


1 


0 


0 


0 


92601 


0 


0 


0 


0 


0 


0 


926 


0 


0 


0 


0 


0 


0 




42 


3 


21 


1 


1 


16 
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TABLE A-7 

DISTRIBUTION OF TIME SPENT IN INDEXING OF THOSE DOCUMENTS 
RETRIEVED FOR THE RETROSPFCTIVE SEARCHES RUN 



Search 


Docs 


Summation of In 


dexin^[ Time 


by Indexing 


Source Cat 


enory 


No. 




1 


2 


3 


4 


5 


Total 


12603 


14 


2 


55 


6 


0 


50 


113 


12624 


1 


0 


6 


4 


0 


6 


16 


12630 


2 


0 


5 


0 


0 


12 


17 


12636 


1 


0 


5 


0 


0 


0 


5 


22634 


7 


0 


20 


3 


0 


43 


66 


22635 


14 


3 


45 


0 


4 


55 


107 


32528 


3 


0 


7 


0 


5 


18 


30 


32604 


4 


0 


16 


20 


0 


8 


44 


32605 


3 


0 


12 


0 


0 


8 


20 


42543 


5 


0 


17 


8 


0 


32 


57 


42637 


2 


0 


0 


0 


0 


4 


4 


62504 


1 


0 


0 


3 


0 


0 


3 


Q2SQ4 


1 


0 


4 


u 


0 


3 


7 


92601 


1 


0 


10 


0 


0 


5 


15 


92624 


2 


0 


14 


0 


0 


5 


14 




- 


5 


216 


44 


9 


249 


518 
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SDI 

RKOVKSTKR INDKX 



RFQrjKSTKR 


QRGANI/A nON 


AuAir, A \I 


AFML/LL 


Allniikov S D 


AFMF/MXE 


Anrle r son , C S 


AFML/LTF 


Anspdch, \V F 


AFML/LNE 


Arrioid, F K 


AFML/i\lBP 


A rnson H F 


AFML/LLS 


Askms, D. R 


UDRI 


Aunian. G W 


AFML/LTE 


Bentley, F F 


AFML/LP 


Ben/., R S 


AFML/LTF 


Bertke, R S 


UDRI 


Bialrt. M. 


AFML/LTE 


Blakeslee, H W 


Franklin Institute Research Lab 


Bay n ton T A 


AFML/LTE 


Browning. C F 


AFML/MBC 


Buckley, M J. 


AFML/LL 


Campbell, G L 


AFML/LTM 


Champa, R A 


AFML/LPH 


Clark. L 


AFML/LTM 


Cohen B. 


AFML/MXA 


Corblv D M 


AFM T / T T N 

JC iVl JLj / JLj JLj IN 


Crane, R L 


AFML/LL 


Crawford, W J 


AFML/LPA 


Crosby. J J 


AFML/LL 


CunninghaiTi, A. 


Lockheed -Georgia Co, 


Davidson, ,T K 


UDRI 


Davis, K. A 


AFML/LN 


Davis, S 0 


AFML/LL 


Denman. G L 


AFML/MXS 


Denson, D D, 


AFML/MBP 
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REQVESTKR 


ORGAM/ATIO\ 


DePierro, V 


AFME/EL 




afml/lp 


Donlan. V" 


AFML/LPE 


Drz^^] L T 


AF\ti7L>:X 


Dupweke, P W 


EDRI 


Duval]. J) 


UDRI 


EktTian, W J. 


AFML/MBP 


Emrich B R. 


ASD/YHEF 


Engle, A G. 


UDRI 


Fvers, R C. 


AFML/MBP 


Ezekiel, H M 


AFML/LNF 


Farmer, R W 


AFML/MBC 


Fiscus, I 


UDRI 


Fredericlr. V/ G D 


AFME/LPE 


Fujishiro, S. 


AFML/LL 


Garrett, H J 


AFML/LTE 


Gehatia, M T 


AFML/MBP 


Geisendor fer , R F. 


AFML/LL 


Glenn, G. M. 


AFML/LTM 


Gloor, W. H. 


AFML/ LN 


Goldberg, W 


AFML/LP 


Goldfarb. L J 


AFML/MBP 


Grandt, A F 


AFML/LL 


Grant, R. 


UD 


Graves, R 


UDRI 


Haggard, D K 


AFML/LL 


HalK J A 


AFML/LL 


H.i rnic r, R 3 


UDRI 


Haury, G E. 


AFML/LPH 


Headrick, R. E. 


AFML/MBE 


Hecht, N. 


UDRI 
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REQUESTER 



ORGANIZATION 



lleiii ru!). J P. 
HelnuJuak, T. E 
Henien^e r , P M 
HtMide r fjn , J P 
Hickniott. J. P. 
HoUenberg, G 
Hopkins, A K 
House. P i.. 
Hutciiens , 
IHer, W J 
Iglauer, N 
Jerina. K. L, 
Johnson, W P 
Jumper, G 
Kennard, R 
Kirkpatrick, N B 
Klarquist, N E 
Knight, M 
Koenig, J. R 
Kopell, L 
Kuhl, G E. 
Lee, T. 
Lehn, W L 
Leinberger, K. 
Litnak, S 
Lopez, A. 
Loughran, G. A 
Lyon. S P 
McDevitt, i\. T 
Marcus, H. 

X4aterial Science Corp. 



/ FAIT/ EPF 

AFML/\TE^P 

AFME/LPrJ 

Ai^\iE/L L 

AFiML/FPF 

AFlVi E/FI 

AFML / FPH 

AF\tL^MXE 

AFML/EPE 

A F ML /MX A 

AFML/LP 

AFA4L/MBE 

AFML/LNE 

AFML^MXS 

AFML/LTM 

AFML/LL 

AFML/LLM 

AFML/MXE 

AFM L/MXS 

AFML/ LTM 

AFML/LP 

AFML/LPH 

AFML/MBE 

UDRI 

AFML/LTN 

AFML/LTN 

AFML/MBP 

AFML/LL 

AFML/LP 

AFML/LPT 

Material Science Corp. 
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ERJC 



REQUESTER 


ORGANIZATION 


May, D. R 


AFML/LN 


Metzger, G. E. 


AFML/LL 


Meulamans, J L 


AFML/LTE 


Meyer, F H. 


AFML/MXA 


Mild rum, H. 


UDRI 


Morris, G J 


AFiML/LN 


Morrissey, E J. 


AFML/MXE 


Neff, R. 


AFML/LC 


O'Hara, W. 


AFML/LTP 


Olson, J C 


AFML/LPE 


Opt, P c. 


AFML/LN 


Parr^sh, P. 


AFML/LL 


Peters, L. J 


AFML/LTM 


Pierce, B. J. 


AFML/LPE 


Pierce, C. M. 


AFML/LL 


Powell, W R. 


AFML/LP 


Poynter, J. W. 


AFML/LLS 


Pratt, C. A. 


AFML/MXS 


Ramke, W. G. 


AFML/MB 


Ray, J. D 


AFML/MBC 


Reimann, W. 


AFML/LL 


Reinhart, T. 


AFML/MBC 


Rhodehamel 


AFML/MXE 


Rice, D. A. 


AFML/LL 


Rolinski, E. J. 


AFML/LP 


Rondeau, R. E. 


AFML/LPH 


P do, ^ 


AFML/LPH 


Ro. xg, Harold 


AFML/MBP 


Rosenbe rg, Herbert 


AFML/LPH 


Ross, J. H. 


AFML/LN 


Rowand, P. R. 


AFML/LL 
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ORGANIZATION 




VDRl 




AF.\4I7LL 


Rus-^o. " J 


AFM L/LC 


Rutiuw , K 


AI MI /LP 


Rv.m, M '[ 


AFML/MBP 


Sa ia('<'ii , R i 


AFML/DOP 


ru'hir.idl, D 


AFML/MBC 


Scfi>vjtu G r 


AFMl7iMRE 




AFML/LN 


Schwartz, fl S. 


AFM L/ MB 


Schwcnker, H 


AFML/LN 


Shil],tr>, K P 


AFML/LLP 


Slum mm, K D 


AFML/LL 


Shinn. D A 


AFML/MXA 


Simpson, R. P 


AFML/LLP 


S^inlh, C F 


AFML/MBP 


Smyth, R R. 


AFML/LLD 


Snyder, C E, 


AFML/LNL 


Srp, C. O 


AFML/LL 


Standage, A 


UDRI 


Stanton. R. M. 


AFML/LN 


Starks, D 


AFML/LTN 


Stevison, D F. 


AFML/LP 


Strang, J R. 


AFML/LN 


Sidlivan. J. J. 


AFML/MXE 


Tamborski , C 


AFML/MBP 


Tanis, C 


AFML/LTN 


Tanner, H A. 


AFML/LPE 


Tarrants, E. H 


AFML/LTE 


Tesson, J. T. 


AFML/MXS 


Tolley, L. G 


AFML/LPH 




:<I iM'l STER 
i'r!ii,-;e. H K 
'i b.i I , S W 
Yciblaiek, F 
Voss. D. K. 
Vods, D P 
V\ lie eier , hZ. 
Wheeler, W. H. 
\V'illi?nison, J R 
VV:nn, R. A 
\^"ittebort, J I. 
Watn:an, R. E. 
Zitiiniern-ian , P. 
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ORGANIZATION 

AFML/ETE 

AFML/CA 

AFNIE/MXS 

AFML/MBC 

AFML/LL 

AFML/LTN 

AFML/MXS 

AFML/LTN 

AFML/EP 

AFML/LTE 

AFML/MXE 

AFML/MBE 



APPENDIX C 
SDI PROFILE TOPICS 
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s;h sr;:vi<c!! ri^.qi.'ests processed 

; DECEMBER 1971 •• 30 NOVEMBER 1972 



: 0. 


SP:/iRCH TITLE 




Organic Fluorine Compound 




Ferrocene , Compounds, Metallocene Polymer s 




Spirc^polymer s , Spiro com pounds 


M ' ^ } 


Damping, Flotation Fluids 




Properties uf High Temperature Polymer Comp 




Testing of Polymer Composites 




Process of Polym.er Composites 


9*^037 


Transparent Films for Windows 


}9041 


Cleaning of Aircraft 


99046 


Transparent Materials 


g904[< 


Radar Absorbing Materials 


99070 


Aircraft Armor Materials Impact 


99071 


Carbon Fiber Research/Technology 


9907Z 


Three Dimensional Fibers 


990^2 


Environmental Effort on Fibrous Materials 


99083 


Fabric Properties 




Flammability of Materials Fabrics 


99085 


Recovery, Safety of Personel 


99086 


Parachute System -Loading 


99087 


Expandable Structures 


99088 


Coated Fabrics 


99089 


Properties, High Strain Rate-Fibers 


99094 


Fiber Optics 


90095 


Electrically Conductive Fibers 


99099 


Compressor Blades for Aircraft Engines 


99112 


Forming of Metals 


99117 


Powder Metallurgy Techniques 
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SEARCH TITLE 


99128 


Paints, Primers, Surface Finish 




Poiynie r C orx'posite T ankage 


991 3S 


B att eries - Materials 


991^.1 


Electrical Powder Devices, Electro - Clieiiiical 


99152 


Radomes T-IicrH T" pmrspfatn tp F)i r^l ppt "ti r* q 


9916 1 


Fluoro Organic Compounds 


99i6S 


Fluo rinated Polym ers 


99169 


Melting of Metals h. Alloys 


99175 


Klathematical x'^nalysis of Metal Working 


9917 7 


Temperature Measuring Instrumentation 


99179 


Pol va c r vl oni t ri 1 e — Dpmrnnnsiti nn pTnHiiptinn 


99180 


Rain Dust Erosion PhenoiTiena 


99182 


High Temperature Ceramics 


99183 


Ceramic, Metal Composites 


^^91 84 


Orthopedic Implant Materials 


Q9197 


Crystalline Carbon Fibers, Thermal Analysis 


99198 


Rare Earth Alloys Crystal Structure 


99199 


Rare Earth Co Magnetic Materials 


99200 


Holography Crystal Deformation 


99201 


Ceramic Coatings, Flame Spraying 


99202 


Mechanical Properties of MgO Glasses 


99203 


Design of Instrumentation 




Gas Chromatography Decomposition of Pol^Tiiers 


99233 


Ceramic Substrates Packaging for Magnetic Devices 


99237 


Energy Conservation Materials 


99238 


Masers and Lasers 


99239 


Luminesence, Optical Prope rty Special Materials 


9-1243 


Metal Processing 


99254 


Molecular Vibration Spectra of Materials 


99255 


Instrumentation for IR Spectra 
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SEARCH 

NO. SEARCH TITLE 

99257 NonnietaUic Radomes Fabrication 

99259 Decomposition of Polymers 

99287 Joining, Welding of Metals 

99288 Metal Composites 

99289 Carbides, Cermets Phase Diagram 
99299 Fibrous Materials for Clothing 

99301 Processing of Ablative Composites 

99302 Ablation, Ph enomena Mechanism 

99303 Adhesives - Properties and Interfacial Phenomena 
99305 Effect of Electrical Field on Interfaces 

00306 Composites 

99317 Properties of Aerospace Materials 

99318 Synthesis of Perfluoro Compounds 

99319 Ozone Chemistry 

99320 ^^^2 O^i^^tio" of Perfluoro Materials 

99321 Cvclic Organic Peroxides 
99324 Coating Wear and Erosion 
99329 Superconductivity 

99342 Metallic Composites 

99345 Solid State Electronic Materials 

99346 Physical Chemistry 

99347 Ceramics 

99350 Laser Radiation on Materials 

99351 Viscoelasticity and FractUre 

99354 IR Seaming Devices 

99355 Semiconducting Glasses 
99357 Differential Thermal Analysis 
9936'^ Electron Microscopy 

9*^ 3r>' Laser and IR Windows 

99363 Cadm ium Telluride and Zinc Selenide 

99365 Microstructure, Mechanical Properties, Working 
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c r A O T I 

^xL AK(. i\ 




InU . 


SEARCH TITLE 




Heat Treatment of Titanium 


7 \^ -> D / 


Powder Metallurgy 


99368 


Qualitative Microscopy 


99 374 


Mechanics of Metal Composites 




Synthesis of Hydraulic Fluids 


r> o o o o 


Functional Laser Trimming 


7 ^ J o -5 


UHF Broadband Amplifiers 


O O "2 O '2 

99 >9 3 


Polymeric Protective Coatings 


99 394 


Erosion 


o o oo c 

9939^ 


Directionally Solidified Eutectics or Coniposites 


oo oo Q 

9939o 


Aluminum - Chromium Binary 




Liquid Lubricants 


994U^ 


Fatigue, Mechanical Properties of Aluminum and Steels 




Solid Lubricants Compacts 


O O y1 O /I 

994Uo 


Rhenium Ductilizing of Tungsten 


99407 


Solid Solution Softening BCC Metals 


O O >l o o 

99408 


Ablation and Ablative Material 




Laser Window Materials 


yy^c ^ 


Direct Solidified Eutectics 




Transformations in Ti Ni Co Nb 


oo /I 5 "7 

994Z7 


Corrosion Data 


99428 


Electrodeposition 


O O /I o o 

99429 


Storage Material Capabilities 


99432 


Analysis of Polymers 


99435 


Magnetic Materials and Properties 


99440 


Powder Metallurgy Technology 


99441 


Glass Fabrication 


99442 


Metal Polymer Interfaces 


99443 


Polymer Composite Interfaces 


99444 


Polyphenolquinoxaline Resins 


99445 


Polymer Degradation 
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SEARCJJ 



NO. SEARCH TITLF 

97446 Working, Alloy Development of Ti, Al Alloys 

99447 Structure of Perfluoro Organization of Fluoro 

Metallic Compoiinds 

99448 Corrosion and Embrittlement of 4340 and D6aC 

99449 State of the Art of Epoxy Polymers 

99450 Chemical Types, Curing of Epoxies 
99452 Magnetic Resonance 

9*^45 3 Acoustical and Optical Radiation 

99454 Compatibility of Metals 

99455 Structural Application of Metals, Composite 

99456 Fabrication of Metals Composites 

99457 In-Service Corrosion Failure 

99458 Lubricants for Aerospace Systems 

99459 Thermoplastics 

99461 Advanced Composite Application 

99462 Transparent Materials 

99463 Lasers - Materials and Effects 

99464 CO^ Lasers 

99465 RAF Magnetic Materials 

99466 Elastomers Sealants, Polymers 

99467 II-VI Semiconductors 

99468 Laser Damage on Materials 

99483 Tooling for Composites 

99484 Organic Compounds-Nuclear Magnetic Rasonance 

99485 Liquid Crystals 

99486 Laser Effect on Materials 

99487 Laser Effect on Materials 
99506 Al Composites with Boron Fibers 
99S0 Fracture Theory of Metals 

99519 Properties of Rigid Polymers 

99520 Metal Matrix ComposHes 
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SEARCH 



NO 


oiLAR'wH TITLE 


99s? J 


Phase Transformation of Defects 




Joining Oxides of Metals Alloy Development 


7 7^5^ J 


Hydraulic Fluids and Lubricants 


/ / ^ fc» T 


Polyineric Protective Coating 


77 7) C\J 


Rain and Dust Cloud Simulation 


/ 7 _/ t* \J 


Ihermal Protection Systems 


7 7 _/ 4* / 


Inorganic Nonmetallic Reinforced Fibers 


7 7 fc»0 


Polymeric High Strength Fibers 


99529 


Stress Corrosion, Cracking 


99530 


Shock Phenomena 


77l? j1 


Cutting Tools Ti and Superalloys 


QQC -3 9 

7 7^ 


Fluids and Lubricants 


7 7^ 


Oxidation and Coating of Metals 


7 7-/ JU 


Mechanical Properties, Testing 


99537 


ohells, Panels - Structural 


7 7 J «50 


Acoustical Effect on Materials 


7 73^7 


Temperature Effects on Microstructure 


7 7-? T^Vy 


Dynamic Loading Behavior of Materials 


QQC /ll 
7 7-> T^l 


Bearing Systems for Space 


7 7-> 


Testing Lubricant - Bearing System 


QQC^ 
77-5^ J 


Physical Metallurgy 


77-? 


Ladder, Spiro, Thermal Stability Polymer 


QQC At; 

7 71? rtD 


Nonflammable Fibrous Materials 


99546 


High Strength Polymer Fibers 


99547 


Composite Reinforcements 


99548 


Heat Flow in Fibrous Materials 


99549 


Liquid Fuel Fires 


99550 


Laser Hardened Materials 


99551 


Math, Statistic, Prediction of Behavior 


99552 


High Temperature Plastic Coatings 
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SEARCH 



NO. 


SEARCH TITLE 


99553 


Composites Data 


99554 


Thermal Protection Systems for Rockets 


99555 


Laser Window Materials 


99556 


Crack Initiation at Notch 


99557 


Joining Welding, Brazing 


99558 


IR Laser Window Materials 


99559 


Metal Composites Height and W^eight 


99560 


Emission From Material Under Stress 


99561 


Elastomers and Applications 


99562 


Reinforced Polymer Composites 


99563 


Stress Corrosion Kinetics 


99564 


Mass Spectrometry 


99565 


Radar IR, UV Absorption Materials 


99566 


Optical Properties of Inorganic Materials 


99567 


Structural Adhesives 


99568 


Surface Analysis 


99569 


Aerothermodynamics 


99570 


Reaction Kinetics 


99571 


Thermodynamics 


99572 


Chemical Physical Behavior in Ablative Wakes 


99573 


Wear Properties of Titanium 


99574 


Fracture Mechanics 


99575 


Mechanical Fasteners 


99576 


Titanium Alloy Properties 


99577 


Vacuum Deposition Techniques 


99578 


Polymeric Dielectric Coating 


99579 


Measurement of Optical Properties 


99580 


Energy Effect on Materials 


99581 


High Temperature Corrosion Protective Coating 


99582 


Paint, Coating Formulation Camouflage 
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SEARCH 





SEARCH TITLE 




Properties of Ni SuperaUoys 


C%C\C O A 


Infrared Detectors, Photoconductivity 


QQCQd 


Amplification of Surf ace Acoustics 


QQ CQL 

VVboo 


Elastomeric Fluid Seals 




Rolling Technology of Metals 




Manufacturing Technology Steel Ti Al Mg Be 


QQCQO 
770O7 


Nondestructive Testing, Quality Control 




Design Behavior of New Composites 


QQ t;Q1 


Effect of Laser Radiation on Materials 


7 7-57^ 


Welding of Titanium Alloys 


oo c;o '3. 
yyoy 5 


Oxidation of Ni SuperaUoys 


7 73 7^ 


IR detectors - amplifiers 


QQ^Q^ 

7/3 73 


Ceramics - Properties and Application 


O O C O 


Lubricant Composites with Titanium 


^7397 


Thin Metal Foils - Preparation 


7739o 


Powder Alloys Ti Al Co Ni 


C\C\CiC\C\ 

yyoyy 


Temperature Control Coatings 


n n A A 

y 9500 


Rain Resistance 


77DU1 


Carbon Fibers - Pyrolysis of Organic Fibers 


rt n A o 


Weldability of Titanium Alloys 


yyb0 5 


E M Windows IR Laser Radiation 


99504 


Fuel Tank Sealants 


A A / /\ C 

99605 


Ferroelectric Materials 


99606 


High Temperature Application of Materials 


99607 


Coating Processes 


99608 


Theory of Metal Plastic Deformation 


99609 


Microwave Ferrites 


99610 


Properties of Textiles 


99611 


Optical Contaunination of Spacecraft Surfaces 


99612 


Ultrasonic Testing 


99613 


Fracture Mechanics 
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SEARCH 



NO. SEAPCIi TITLE 

99614 Semiconductor Materials; Properties 

99615 Semiconductor Compounds 

99616 Garnets Ferrites and Computers 

99617 Fabrication Process - Electronic 

99618 Electro-Optical Materials 

99619 Thermionic Tubes - Materials and Proces 

99620 Dielectr onic for Electronic Devices 

99621 Epoxy Graphite Composites 

99622 Coating Vs Erosion 

99623 Forming Techniques 

99624 Properties of Composites for Missiles 
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APPENDIX D 
DEFINITION OF SUBJECT CATEGORIES 



AMIC COSATI 
01 01 



02 



03+04 



03 



06+07 



CATEGORY 

Aeronautics 

Aerodynamics 

Aeronautics 

Aircraft 

Aircraft flight control and instrumentation 
Jet engines 

Astronomy, Astrophysics, Atmospheric Sciences 

Astronomy 

Astrophysics 

Atmospheric physics 

Meteorology 
Chemistry, Biology, Medical Sciences 

Biochemistry 

Bio 2ngi nee ring 

Biology 

Chemical analysis 
Chemical engineering 
Inorganic chemistry 
Life support systems 
Organic chemistry 
Physical chemistry 
Radiochemistry 
Toxicology 
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AMIC COSATI 
04 09 



05 



06 



07 



08 
09 

10 
11 



12 

13 
14 
15 



llA 



llA 



IIB 



lie 

IID 

HE 
IIF 



IIH 

HI 
IIJ 
IIK 



CATEGORY 
Electronics and Electrical Engineering 
C omponents 

Electronic and electrical engineering 

Telemetry 
Adhesives 

Ceramic cements 

Organic resin adhesives 

Potting compounds 
Seals, Sealants 

Ceramic-metal bonds 

Mechanical seals 

O- rings 

Ceramics, Refractories, Glasses, Minerals 
Borides 
Carbides 

Carbon, graphites 

Mixed oxides 

Nitrides 

Single oxides 

Coating, Paints, Oxide Films 

Composites Materials, Laminates, Sandwich 
Structures, Honeycomb 

Fibers, Textiles, Cloth 

Metallurgy, Metallography 

Alloys 

Metals 

Oils, Lubricants, Heat Transfer Fluids, Greases, 
Hydraulic Fluids 

Polymers, Plastics 

Elastomers 

Cleaning Compounds, Surface Active Agents 
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AMIC 
16 
17 
18 



COSATI 

IlL 

21 

13 



19 



14 



20 



18 



CATEGORY 
Wood and Paper Products 

Fuels, Propellants, Propulsion Systems, Explosives 
Mechanical, Industrial, Civil and Marine Engineering 
Civil enginee ring 

Construction equipment, materials, supplies 
Containers and packaging 
Couplings, fittings, fasteners, joints 
Industrial processes 

Machining, tools, machine elements such as 
bearings, gas lubrication systems 

Marine engineering 

Pumps, filter^;, pipes, fittings, tubing, and 
valves 

Safety engineering 
Structural engineering 
Methods and Equipment 
Apparatus 
Detectors 

Laboratories, test facilities, and test equipment 
Recording devices 
Nuclear Science and Technology 

Fuel elements; fuel, nuclear 
Nuclear explosions 
Nuclear power plants 
Nuclear reactors 
Radiation shielding 
Radioactive wastes 
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AMIC COSATI CATEGORY 

21 20 Physics 

Acoiistii. 
C rystallography- 
Electricity and magnetism 
Fluid mechanics 
Masers and lasers 
Optics 

Particle accelerators 
Particle physics 
Plasma physics 
Quantum theory 
Solid mechanics 
Solid-state physics 
Spectrometry, spectroscopy 
Thermodynamics 
Wave propagation 

22 10,16, 22 Space Technology and missiles 

Astronautics 

Energy conversion, solar cells 

Launch vehicles 

Missile technology 

Re-entry vehicles 

Rockets 

Satellites, artificial 
Spacecraft 

Trajectories and re-entry 
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TABLE E-1 

DISTRIBUTION OF INPUT DOCUMENTS BY SUBJECT CATEGORY 



a a T I 1 ^ £k rf-x r 




Documents 






No. 


% 


Ul 




112 






1. 7 






71 






1.1 


U J 




1341 






20.1 


u 




6 






1.0 






30 






0.4 


(Jo 




40 






0. 6 


07 




163 






2.4 


r\ Q 
UO 




125 






1.9 


09 




220 






3. 3 






42 






. 0. 6 


1 1 




1089 






16. 3 


12 




192 






2.9 


13 




136 






2.0 


14 




46 






0. 7 


15 




10 






0.1 


16 




20 






0. 3 


17 




126 






1.9 


18 




324 






4.9 


19 




330 






4.9 


20 




384 






5. 8 


21 




1581 






23. 7 


22 




229 






3.4 
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RETROSPECTIVE SEARCH REQUESTS PROCESSED 
1 DECEMBER 1971 - 30 NOVEMBER 1972 

SEARCH 



NO, S EARCH TITLE 

2655 Convection Transfer 

2656 Critical Strain Grain Growth 

2657 Rare Earth Ni-Co-Fe Alloys 

2658 Wear Fretting of Titanium 

2659 Flexural Testing of Laminates 

2660 Rare Earth Cu-Ni-Fe Alloys 

2661 Rare Earth Co-Fe-Ni Alloys 

2662 Thickners- Liquid Hydrocarbon Fuels 

2663 Cost of Titanium 

2664 Documents on Titanium Ni- Alloys Eutectoid 

2665 High Temperature Nickel Superalloys 

2666 Iodoform Heat of Vaporization 

2667 Inorganic Polysulfides 

2668 Polysulfide Preparation 

2669 Alloy 713-C Mechanical Properties 

2670 Inconel 71 C High Temperature Fatigue 

2671 Thermochromic Compounds (25-50'^C) 

2672 Testing Adhesives Bonding Joints 

2673 Silver Bearing Corrosion Inhibition 

2674 Surface Finish of Aluminum 

2675 H^ Effect on Titanium- Al-Sn-V 

2676 Hydrogen Pickup of Titanium Alloys 

2677 Weldbonding Aerospace Structures 

2678 Mechanical Fasteners for Aircratft 

2679 Fretting Corrosion 

2680 Fatigue of Incoloy 901 
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SEARCH 




NO. 


SEARCH TITLE 


2681 


Hot Deformation of Alloys 


2632 


Ballistic Impact Testing 


2683 


Iinpact Testing of Composites 


2684 


Composites Research and Development, AFML 


2685 


Graphite Fiber Reinforced Composites 


2686 


Mesophase From Pyrolysis 


2687 


Stress Corrosion Cracking of Titanium Alloys 


2688 


Powder Metallurgy of Titanium and Titanium Alloys 


2689 


Light Laser Beam Choppers 


2690 


Graphite Fiber Processing 


2691 


Hot Deformation on Alloys 


2692 


Textured Titanium 


269 3 


Phosphine Oxide Polymers 


2694 


Phosphor is Containing Polymers 


2695 


Ultrasonic/Defect Interactions 


2696 


Adhesive Bonding Beta III 


2697 


UV Stabilizers for Polymers 


2598 


Low Outgassing Polymers 


2699 


Hydroforming 


2700 


Ultrasound Attenuation 


2701 


Tooling for Composites 


2702 


Metal- Polymer Interfaces 


2703 


Polymer-Composite Interfaces 


2704 


Polyphenolquinoxaline Re.;ins 


2705 


Polymer Degradation 


2706 


Corrosion and Embrittlement of 4340 and D6 


2707 


Structure of Perfluoro Organofluoro Metallic Compounds 


2708 


State of the Art of Epoxies 


2709 


Chemical Types - Curing of Epoxies 
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SEARCH 



NO. SEARCH TITLE 

2710 Working, Alloy Development of AUTi Alloys 

2711 Perfluoro Aldehydes or Ketenes 

2712 Reflective Coatings for Air Materials 

2713 CO^ Laser Photometry 

2714 Titanium Fires in Aircraft 

2715 Rare Earth Magnetic Materials 

2716 Prestress Effect on Fatigue Life 

2717 Elastomers, Sealants, Polymers 

2718 II - VI Semiconductor 

2719 Organic Fluoride Compounds 

2720 Organic Compound NMR 

2721 Liquid Crystal Display Devices 

2722 Zinc Selenide - Heat Conductivity 

2723 Al Composites with Boron Fibers 

2724 Fracture Theory of Metals 

2725 Instability Fractures 

2726 Nitroso Elastomers 

2727 Fasteners for Composites 

2728 Thermal Shock Behavior 

2729 Oxidation, Sulfidation of Metals 

2730 Polyphenylquinoxalines 

2731 Corrosion Inhibitors 

2732 Laser Hardened Materials 

2733 Electric Field Controlled Heat Transfer 
27 34 Oxidation of Niobium 

2735 Reinforced Thermoplastics 

2736 Service Life of Ni Superalloys 

2738 Stainless Stee' 13-8 

2739 Properties Bihavior Ni Superallc ys 
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SEARCH TITLE 




Hydrogen Enibr:« ttlement, Ferrous Alloys 


2741 


Deformation Mechanics of Ni Superalloys 


274Z 


Microstructvre of Ni Superalloys 


274"' 


Titanium- A huninurn Alloys 


Z7 44 


Historica] Analysis Materials Development 


Z7IS 


Deconiposition of Polymrethanes 


2746 


II - IV Compounds Properties 


2747 


F-Ivl Generation Acoustin Waves 


274^5 


Structural Adhesives 


2749 


Metal Surface Preparation 


27=^0 


Kpoxy and Polyn^tide Resins 


27^ 


Trans- ""rent Materials 


27^2 


Carbon Foams 


27-^^3 


Chemical Analysis of Fuel Tank Residue 


2754 


Rigid Mullite 
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